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Abstract: This article reports findings on a case study of Ellie as she participates in a language arts curriculum 
that incorporates multimodal literacy practices—including photography, drama, and art—to teach reading 
and writing. Our study was informed by the theoretical framework of multimodal social semiotics, which 
provides insight into how mediational tools allow for greater complexity of thought while engagement within 
the modes expands the potentials for learning. Multimodal interaction analysis (Norris, 2004) was applied to 
video data from this study. Findings reveal the significance of using photographs for Ellies meaning making 
in the third grade classroom. In addition, we were able to observe how Elbe’s responses at times conflicted 
with the intentions or expectations of the classroom. This study reveals the significance of multimodal video 
data analysis as a way of understanding the complexity of literacy practices in the classroom. 
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T his article reports findings on a case study of 
Ellie, a third grade student, as she 
participates in a language arts curriculum 
that incorporates multimodal literacy 
practices, including photography, drama, and art to 
teach reading and writing. 1 Our interest in 
researching a multimodal language arts curriculum 
is to consider how students use expanded modes of 
learning in the classroom. Incorporating multimodal 
instruction has the potential to provide more 
opportunities for students to build and share 
knowledge, particularly if they struggle with print- 
based literacies. As literacy researchers, we believe it 
is important to consider how multimodal 
instruction could allow for new ways that students 
engage in the language arts curriculum. Ellie stood 
out as an illustrative case because she was especially 
reflective of how her photographs helped her 
writing, yet she was identified by her teacher as 
“struggling.” As we analyzed her literacy learning in 
the classroom, we found that her participation in 
reading and writing practices was complex and that 
visual strategies supported her literacy learning. Our 
primary question is this: How does the integration 
of multimodal instruction affect how a third grade 


1 1 acknowledge that there is a gender spectrum and that 
myriad pronouns exist that I can use when referring to 
individuals in my writing. Throughout this article I will 
use “he” to refer to individuals who identify as male, “she” 


reader and writer responds and learns in the 
language arts classroom? 

Theoretical Framework 

The analysis of Elbe’s participation in a multimodal 
language arts curriculum rests on her connections 
and responses within the classroom context. In this 
section, we will explain how the theoretical 
framework of multimodal social semiotics informed 
our understanding of this case study. 

Multimodal Social Semiotics 

Elbe’s language arts curriculum featured multimodal 
projects with a specific focus on photography as a 
means to communicate and learn. In multimodal 
social semiotics, social actors rely upon available 
modes to create meaning; the potential meaning 
created is directly inbuenced by the availability of 
resources and their related affordances and 
constraints (Halliday, 1977). A multimodal 
curriculum refers to the idea that many modes, or 
systems of expressing meaning, are used to build 
knowledge (Kress & van Leeuwen, 2006; Serafini, 
2012). Multimodal social semiotics also describes 
learning as a process that occurs collaboratively with 

to refer to individuals who identify as female, and “ze” for 
individuals who identify as gender-neutral. I have 
selected these pronouns because I believe they are more 
familiar for a diverse audience of readers. 
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others and is affected by social contexts (Halliday, 
1977; Lenters, 2016). 

While all texts can be considered multimodal (e.g., a 
child’s written story might reflect ideas from her 
imaginary pretend play she did at recess or a 
television show she watched at home), a multimodal 
pedagogical approach that intentionally integrates a 
variety of communicative modes expands options 
that students have for both learning and expressing. 
Using a range of mediational tools allows for greater 
complexity of thought while engagement within the 
modes expands the potentials for learning. Research 
has documented how emergent readers and writers 
have learned and even situated themselves as 
experts through tools (e.g., maps, visuals) made 
available with semiotic resources (Kissel, Hansen, 
Tower, & Lawrence, 2011). For example, Cappello 
and Lafferty (2013) found in their research on 
integrating photography in an elementary classroom 
to promote disciplinary literacy that students’ 
responses demonstrated an “alternative language” 
that reflected complex understandings across 
subject areas. Specifically, the teacher and 
researchers observed how students’ metacognition, 
risk-taking, and reflective thinking were important 
aspects of how they responded to the multimodal 
curriculum. 

Encouraging multimodal responses (e.g., products 
combining visual and linguistic modes) is one way 
educators have developed pedagogy that encourages 
more expansive approaches for learning, thus 
creating more inclusive classroom contexts 
(Cappello & Hollingsworth, 2008; Siegel, 2006). In 
addition, incorporating multimodal approaches to 
learning may be more engaging for students because 
it often builds on their out-of-school literacy 
practices (Kyser, 2013). Opportunities to mediate 
understanding using what could be considered 
traditional production (e.g., linguistic 
communication) alongside the incorporation of 
multimodal elements (e.g., photographs, textiles) 


has the potential to support students’ literacy skills 
(Bomer, Zoch, David, & Ok, 2010). 

Methods 

This study investigated a third-grade student in her 
natural, classroom environment and relied upon 
data sources consistent with case study research 
(i.e., field notes, interviews, and artifacts) (Dyson & 
Genishi, 2005; Yin, 2009). In the following sections, 
the context, participants, data collection, and data 
analysis are described in detail. 

Context and Setting 

This article represents analysis of part of a larger 
study that took place in an elementary school that is 
a magnet for the arts and humanities (Wiseman, 
Kupianinen, & Makinen, 2016). In this study, Angela 
documented a third-grade classroom in a diverse 
urban public school that integrates photography 
into the language arts curriculum as a central 
component of reading and writing instruction. In 
the classroom, there were 22 students: 8 males and 
14 females; the students identified their racial 
background as Hispanic (4), White (7), Black (10), 
and Multi-racial (1). Five of the students received 
services because the school identified them as 
Academically Gifted and four received services for 
English Language Learning. The teacher reported 
that twenty percent of the students passed their 
state reading test, which is the end of the year exam 
that involves vocabulary and reading 
comprehension questions. 

Ms. Brown (all names in this article are 
pseudonyms), the teacher of this classroom, 
identifies as a Caucasian woman who has taught in 
public schools for more than 10 years. She 
implemented an art-based curriculum called 
Literacy Through Photography (LTP) into her 
reading and writing instruction. Students used 
critical thinking and creative understanding as they 
completed projects related to three themes: self- 
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portrait, proverbs, and community. In each project, 
students worked in groups to develop ideas by using 
photography, drama, art, writing, reading, and other 
methods of understanding. Each of the three 
projects took approximately three to five weeks to 
complete and were integrated in the language arts 
curriculum throughout the school year (see 
Wiseman, Kupianinen, & Makinen, 2016; Wiseman, 
Pendleton, Christianson, & Nesheim, 2013). 

Data that comprised the final project, “My 
Community,” is the focus of this case study. In this 
project, students began by reading books and 
conceptualizing how they might define community. 
They created maps, labeled different features, and 
thought about various modes of representations 
(e.g., pictures, colors, sounds). Students took a 
33mm camera home for one or 
two nights and photographed 12 
scenes representing their 
communities. After studying 
their negatives and discussing 
how their visual representations 
reflected their communities with 
others, students selected one 
picture to develop in the 
darkroom. As they developed 
their pictures, they learned about exposure, light, 
chemicals, and other aspects of image development 
affecting the meaning, tone, and mood of a 
photograph. 

Ellie: An Illustrative Case Study 

We chose to focus on Ellie; she was a student who 
was described by her teacher as “struggling” with 
reading and writing. Ellie identified as an African- 
American female; she was one of many students in 
the classroom who had not passed the state reading 
test and was targeted as a student of concern. She 
did not qualify for any school services and did not 
have an Individualized Education Plan. However, 
the teacher integrated visual learning into her 


“In each project, students 
worked in groups to 
develop ideas by using 
photography, drama, art, 
writing, reading, and other 
methods of understanding.” 


guided reading and writers workshop as 
instructional opportunities to meet Elbe’s needs. 
Ellie was an illustrative case (Creswell, 2013), 
providing us with the opportunity to delve further 
into understanding her meaning-making with a 
multimodal curriculum. 

Ellie was very motivated and engaged in class, but 
she seemed to struggle with articulating her 
thoughts in writing and comprehending reading 
texts. Ellie identifies herself using positive 
language— “friendly, good listener.” Reading was 
her least favorite subject because “it is so quiet. She 
added, “I don’t like quiet.” She enjoyed interacting 
with others and preferred any assignments where 
she could work collaboratively in groups. Ellie 
comes from a family with two brothers and two 

sisters and lives in an apartment 
a few miles from school. Ellie is 
close to her family, and she told 
Angela that if she had to 
photograph her most important 
thing in her life, that “it would 
be [her] family.” As Angela 
documented Ellie across the 
school year, it became clear that 
photography was a tool that 
supported her literacy learning, particularly her 
written expression. Ellie explained that photography 
helped her remember details and “add information 
in [her] mind.” She told Angela that when she uses 
photography, “I can add more details. After I looked 
at it and imagined it, I added more details [to my 
story].” 

Data Collection 

In the larger study, Angela applied ethnographic 
techniques of participant-observation and 
descriptive analysis in the classroom setting as data 
were gathered, on average twice a week, throughout 
a full school year (Creswell, 2008; Kamberelis & 
Dimitriadis, 2005). A case study approach was used 
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to provide an in-depth, multi-dimensional 
consideration of this classroom by drawing on 
multiple data sources (Dyson & Genishi, 2003) to 
examine and describe the LTP program in this third 
grade classroom. Data sources included classroom 
observations, student writing and photographs, 
interviews, and discussions. During the photography 
projects, Angela observed and videotaped students, 
collected work samples, and informally asked 
questions about their work. After each of the three 
projects was completed (i.e., self-portrait, proverbs, 
and community), Angela conducted a retrospective 
think-aloud protocol (Schellings, Aarnoutse, & van 
Leeuwe, 2006) with Elbe after each project using 
multimodal interviews. For the retrospective think- 
aloud, Angela sat down with Elbe and viewed the 
various multimodal artifacts for each project and she 
asked open ended questions about how she 
designed the artifacts and how the artifacts reflected 
her understanding of the project. Examples of 
questions include, “What were you thinking about 
when you created this (mode, example 
photograph)?” and “Explain what this (mode, 
example sketch) means? 

Data Analysis 

Data analysis occurred in two different phases. The 
first phase occurred in the larger study and involved 
using descriptive analysis to create a thick 
description of the LTP program, language arts block, 
and classroom routines (Merriam, 2009) and 
thematic analysis to determine the impact of 
multimodal instruction on student learning (Miles & 
Huberman, 1994). Interviews with the teacher and 
students, student work, and field notes were used 
for describing the classroom context (Yin, 2009). 

The themes from the larger study reflected three 
main ideas about student learning: (1) students’ 
options for meaning making expanded when 
working with photographs; (2) student learning and 
engagement increased from integrating students’ 
experiences in the curriculum; and (3) the visual 


medium of photography provided a deeper and 
expansive mode of learning in the language arts 
classroom (Wiseman, 2011). 

After the data collection and initial analysis were 
completed on the larger data set, Angela began the 
second phase of data analysis by collaborating with 
three other literacy researchers who are also the co¬ 
authors of this article using multimodal interaction 
analysis (Norris, 2004; see Wiseman et al., 2014). 
Multimodal interaction analysis is a framework that 
provides a means to capture the perceptions, 
thoughts, and feelings people express as web as the 
attention and awareness levels they exhibit through 
both embodied modes (i.e., spoken language, 
posture, gesture, gaze, and head movement) and 
disembodied modes (i.e., music, print, and layout). 
(Norris, 2004). Discourse analysis, interactional 
sociolinguistics, mediated discourse analysis, and 
multimodality inform this qualitative framework. 
Our purpose for examining the videos using this 
process was to illuminate the complexity of Elbe’s 
learning and understand moments of struggle and 
success. 

Our collaborative data analysis sessions began with 
discussion of significant themes, literacy events, and 
ideas that took place in this classroom. We reviewed 
data from the focal students and discussed how to 
identify segments of video clips for the purpose of 
conducting a microanalysis, which can be defined as 
a more detailed form of coding (see Wiseman, 
Pendleton, Christianson, & Nesheim, 2013). As a 
team, we devised our interpretation of multimodal 
analysis and applied it to three students who were 
identified as struggling with reading and writing in 
the classroom. Elbe was selected because we felt we 
would learn about the complexity of her learning 
through the research methods; we selected five 
episodes that reflected her progression of learning in 
a multimodal LTP project that was about 
community. The episodes included Elbe’s writing 
reflection after an initial brainstorm, her discussion 
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of the negatives she took of her community, her 
description of the picture she developed, her 
elaboration on her community, and her reading of 
the final product after developing her picture and 
writing her story. 

Modes such as proxemics, gaze, posture, head 
movements, music, and language are part of a 
semiotic system that Norris (2004) refers to as 
communicative modes. For the purposes of our data 
analysis, we focused on the modes of gesture (hand 
and arm movement only), posture, gaze, head 
movement and spoken language because they 
seemed most relevant to our study. The modes were 
considered as follows: 

Gesture. Defining gesture as how she moved her 
hands as she spoke, we 
considered how Ellie used 
gestures and noticed that she 
often used gestures when her 
modal density was at its peak. 

This indicated that she was 
using many different ways of 
communication and she could 
be struggling or engaged during 
these times. 

Posture. Posture describes how she positions her 
body in relations to others. An important aspect of 
posture was how posture changes when interacting 
with other people; we could see her lean in when she 
was engaged or move away to break off interactions 
with others. It is important to note that Elbe’s 
engagement in both class settings and interview 
settings was inferred largely through the coding of 
posture. 

Gaze. To understand gaze, we followed Elbe’s 
direction and intensity of eye contact. We also 
analyzed Elbe’s head movement, or the way a 
participant holds her head (Norris, 2004). In the 
majority of the clips, Elbe’s gaze enabled us to make 


stronger inferences about her engagement levels and 
attention foci. 

Head movement. We focused on head movement, 
considering where her attention was directed. Elbe’s 
head movement in our video data primarily 
alternated to focus on image negatives, the 
researcher, her writings, and her peers. 

Spoken language. In addition to non-verbal 
communicative modes, we carefully analyzed Elbe’s 
spoken language. We analyzed transcripts of Ellies 
spoken language in the classroom and during her 
interview session. 

We coded each video clip separately with our 
assigned individual communicative modes, and we 
also wrote research memos on 
observations while coding. We 
ultimately realized that modes 
such as posture, gesture, gaze, 
and head movement played as 
critical a role as verbal modes 
because they carry “interactional 
meaning as soon as they are 
perceived by a person” (Norris, 
2004, p. 2). The clips were presented to the entire 
team for discussion and further analysis. When 
there were differing opinions regarding coding, we 
watched the clips together repeatedly until we came 
to a consensus on the analysis and interpretation. 

In addition to coding modes, we also analyzed the 
clips for both modal density and modal dissonance 
since they represent generative places that either 
reflected sources of struggle or opportunities for 
learning (Norris, 2004). Modal density occurs when 
participants in an interaction draw upon several 
modes at once or draw upon numerous modes in 
quick succession (Norris, 2004). We defined modal 
density as instances where Elbe actively 
demonstrated at least three modes at once. Modal 
dissonance refers to the notion that the 


r 

“We ultimately realized that 
modes such as posture, 
gesture, gaze, and head 
movement played as critical 
a role as verbal modes...” 

C J 
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Time 


41.00-41.05 





Spoken 
Language 
It s a lot of 
sounds but I’m 
not around 
when things go 
down 


Gesture 


beat gesture 
with hand back 
and forth on 
table w f hile 
reading 


Posture 


open; 

standing; open 
to class 
engaged (in 
reading) 


Gaze 


mutual gaze 
(w/teacher) 
and directional 
gaze (at 
teacher) then 
out to class 
then 

directional at 
paper when 
asked to read 


Head 

Movement 
looked dow r n 
w r hile reading; 
directional 


Figure 1. Excerpt from our microanalysis of Classroom Clip demonstrating a modal ensemble. 


communication of the different modes seems to be 
in opposition. 

Researchers' Roles 

All four participants of this research team identify as 
Caucasian women and have had English/language 
arts classroom teaching experiences. We have all 
had classroom teaching experiences and have 
explored ways that we could develop learning 
opportunities for and with students. We have 
particularly been concerned about our students who 
are marginalized, particularly those labeled as 
“struggling” due to linguistic or cultural diversity. 
Our goal is to develop research methodologies that 
acknowledge the complexities of classroom response 


and, rather than approach students as “struggling,” 
promote the idea of understanding students’ literacy 
practices as complex and affected by social contexts. 

Working with a teacher who incorporated 
photography seemed like an opportunity to 
understand how students respond to a multimodal 
curriculum. Our goal was to employ multimodal 
methods of data analysis in order to illuminate and 
explore the complexity of Elbe’s literacy practices in 
the classroom. 

Findings 

During the three different clips that were reported 
in this section, we gleaned information about Elbe’s 
learning, specifically how important it was for her to 


Time 

Spoken 

Language 

Gesture 

Posture 

Gaze 

Head 

Movement 


41.31-41.40 

! 

E: Um, doors 
closing. 


Open 

Standing 
Engaged; more 
so when 
finished and 
facing teacher 

Directional 
gaze(at 
teacher) when 
finished 
reading, then 
partial view of 
unsystematic 
gaze. 

Looked up 
when finished 
Directional 


Figure 2. Excerpt from our microanalysis of Classroom Clip demonstrating a change in Ellies 
communicative modes. 
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use photographs to organize her thoughts. However, 
we found that certain types of questions or feedback 
could affect her ability to participate. In this section, 
we reveal how our microanalysis provides insight on 
her meaning making for this project. 

Sharing her Story: Ellie’s Initial Shut-Down 

The focal lesson began with the teacher leading 
students to consider how multiple interpretations 
can exist within a visual image. In a later interview, 
Ms. Brown explained that the idea behind her 
introduction was to acknowledge the multiple ways 
of interpreting images and ideas and that she 
wanted to “open up the idea of multiple 
interpretations.” This notion of multiple 
interpretations was explored further when the class 
read the book Black Cat (Myers, 1999), in which 
community is described through the eyes of a cat. 

In clip one, Elbe’s full attention is devoted to 
volunteering to read a story she wrote in response to 
Black Cat (Myers, 1999). When the teacher calls on 
Elbe, Elbe looks up, holds her gaze on the teacher 
while she stands, positions herself and her paper, 
then shifts her gaze down as she begins reading. As 
she reads aloud, her posture remains open and 
engaged. Our microanalysis (see Figure 1 ) revealed 
the modal ensemble created by her spoken 
language, gaze, posture, gestures, and head 
movement. We viewed this data as evidence of the 
importance Elbe placed on the opportunity to 
contribute to this classroom. 

Upon completion of her reading, Elbe looks 
outwardly at the teacher. Edward, a student who is 
sitting at her table, leans towards her and says, “That 
sounds like a poem.” Despite Edward’s comment, 
Elbe does not break her gaze with the teacher. Elbe’s 
teacher focuses on a specific response that is more of 
a critique; the teacher responds that Elbe should 
clarify her writing by adding more concrete 
descriptions of sounds in her community, and as a 


result, Elbe’s communicative modes change 
drastically. Her verbal response becomes sluggish, 
abrupt, and minimal (see Figure 2). 

Analysis of the text Black Cat alongside her writing 
demonstrated that Elbe exhibited sophisticated 
writing that mimicked the rhyme and repetition of 
the text by Myers (1999). Her writing matched the 
community theme of the lesson and mirrored poetic 
features of the text. Elbe inserts her perspective into 
this poem, and her voice is strong. She uses 
descriptive words and incorporates language in ways 
that are rhythmic and rhyming. Her use of 
repetition, words such as “round” and “a lot goes 
down,” gives the poem a cadence and flow that have 
dramatic effect. The descriptive phrase “dreaming of 
bright sounds” provides strong imagery of Elbe’s 
interpretation of community; she is even there when 
she is dreaming. When Elbe describes that “a lot 
goes down when nobodys [sic] around,” she alludes 
to how things are happening in her community, 
even when she is not there. Even when she is 
sleeping, Elbe identifies as part of the community 
and feels that she plays an integral role. 

The teacher provided more of a direct or focused 
response. Returning to Halliday’s (1977; 1978) notion 
of meaning potential, it is clear that there were 
parameters to what was an acceptable “braindrain” 
in this classroom. Our microanalysis revealed that 
the manner in which Elbe focused on the language 
and organization of the picturebook actually 
conflicted with her teacher’s intentions for the 
writing assignment. Children access resources 
purposefully in various social contexts that define 
and address certain forms of literacy (Ranker, 2009), 
yet those resources may not match the intended 
purposes of the teacher. While Elbe’s focus when 
she wrote the braindrain was on rhyme and rhythm 
of text, her teacher’s intention was to have students 
incorporate specific visual images from their 
community. Elbe’s approach differed from her 
teacher’s goals for this assignment as her response 
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demonstrated attention to language and rhythm 
exemplified in the picture book. Because the larger 
project was to have students photograph aspects of 
their community, we can see the purpose of focusing 
on concrete images in lieu of poetic elements. 
However, as we watched Elbe’s gaze fall and her 
gestures slow down, we saw that the response from 
her teacher reflected different expectations. Elbe 


During the second part of the project, students 
worked on conceptualizing and describing their 
communities through photographs. During this 
process, Elbe took twelve pictures, brought her 
camera back to school, and selected one negative to 
develop in the darkroom. We selected three separate 
clips from an interview that Angela conducted with 
her in the library regarding her project. 


Time 

Spoken 

Language 

Gesture 

Posture 

Gaze 

Head 

Movement 

8.03-8.07 

and I just think 
that it’s 
interesting. 

Ai: Me too. 

Now, if you 
were going to 
pick three 
words 

Puts her hand 
on her face 
(more posture 
than gesture) 

Open; 

directionally 

facing 

photograph. 
One-two shift 
begins at 8.07 
and completes 
at 8.08 > 

(Directional) 
Looking down 
at picture while 
talking and 
looks up at Ai 
and out at 
something in 
room 

Directional 
shift looking 
up 


8.08-8.12 

to describe 
your 

community 
what would 
they be? 

E: hmmm 

Moves hand 
from mouth to 
in front of body 
(more posture 
than gesture) 

At 8.08, Ellie 
completes a 
one-two shift 
directional 
turn to face Ai. 
She remains 
open and 
engaged. 

(Shifting) 
Looking at Ai 
while she’s 
asking 
question; 
glance at door, 
at right, then 
down at 
picture. 

Directional 
shift looking 
down 


8.13-8.17 

E: well 

behaved, ahh 

Ai: mm hmm 

E: interesting, 
ahh 

No hand 

movement 


Arbitrary 
glance outward 
then looks 
back down at 
picture while 
answering 
question 

Mild 

directional 

shift 


Figure 3. Excerpt from our microanalysis of Photograph Clip Three demonstrating modal dissonance. 


attempted to successfully contribute to the lesson, 
but it seemed to us she was not positioned as a 
successful participant. In this context, her chosen 
way of responding to the picturebook did not fit the 
intentions of the assignment. 

Accessing her Community: Describing 
Photographs and Writing 


Our multimodal analysis revealed how important 
the images portrayed by the negatives were to the 
way Elbe articulated her thoughts. Elbe’s 
movements, gestures, and gaze all indicated that she 
was leaning on these images to respond to questions 
that Angela posed, indicating high engagement in 
the conversations. Her use of gestures increased, 
and she was very animated as she described her 
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photographs. Ellie was clearly impassioned to 
explain her conceptualization of community. The 
meaning making that ensued reflected a connection 
among images, words, body language, and social 
interaction. 

However, when Angela asked very specific questions 
about the project that were devised for the research 
interviews, Elbe’s response was quite different. 
During this section of the interview, Angela asks 
Ellie to list three words that describe her 


community. The matrix above (see Figure 3) shows 
how Angela guided the conversation and how Elbe’s 
responses change. 

During the initial interview in which questions were 
open-ended, Elbe was highly engaged; however, we 
observed a distinct change in her communicative 
modes. Specifically, when Angela asked Elbe to use 
three words to describe her community, Elbe’s 
speech was halted, and other communicative modes 
were disrupted. Elbe looked downwards toward the 
negative and broke eye contact during this question. 
Elbe’s communicative modes demonstrate less 
engagement and decreased participation with 
specifically focused questions and are an example of 
modal dissonance. These specific questions were not 
connected to her ideas or intentions about the 
picture. In many ways, this response to Angela’s 
narrow question and answer request paralleled her 
initial experience as she shared her “braindrain” in 


the first clip. There was a mismatch in the questions 
and Elbe’s own understanding of her community. 

During the different clips that were reported in this 
section, we gleaned information about Elbe’s 
learning. Her motivation, engagement, and learning 
were best reflected through open-ended 
conversations and when she had access to her visual 
images. In other words, the multimodal curriculum 
was providing her with opportunities to express and 
communicate in ways that were productive for her 


learning. However, she was clearly affected by 
directed or specific questioning that limited the way 
that she could conceptualize and respond to ideas or 
concepts. Also, at times, her responses were not 
aligned with expectations of the teacher or the 
interviewer. These connections between 
expectations and her response demonstrated how 
certain expectations from teachers (or interviewers) 
positioned students to be “struggling” (Triplett, 
2007). 

Reading the Community Story 

In the last video clip, Elbe read the final version of 
her community story. At this point, Elbe stood in 
front of both her teacher and classmates to read the 
piece she has composed about her community. The 
act of reading her work aloud to her peers is one she 
took seriously, and the video clip reflects this 
intentional focus. Elbe’s reading reflected high 
modal density (Norris, 2004); our analysis showed 


First draft of community story 

Final version of community story 

In my community it’s a lot around and lots of 
sounds but a lot goes down. But remember it’s a lot 
around and a lot of sound but things go down 
around in my town it’s a lot of sounds but I’m not 
around when things go down and, because I'm 
laying down and dreaming of bright sounds when 

I’m not around a lot goes down when nobodys 
around they’re listening sounds around. 

The Pond. Daddy, can’t we go to the pond? Yes. 

We speed walk there. We don’t want to miss the 
geese. We look around in silence. We see geese 
swimming. We hear waves going through the 
water. We smell dandelions. We imagine the 
spaghetti and garlic bread for dinner. Then we 
started to race to the door. 


Figure 4. Elbe’s first and final versions of her community story. 
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that she was completely focused on reading her 
story, and all modes demonstrated her intensity and 
engagement. Elbe’s final and revised story was as 
follows: 

The Pond. Daddy, can't we go to the pond? 
Yes. We speed walk there. We don’t want to 
miss the geese. We look around in silence. 
We see geese swimming. We hear waves 
going through the water. We smell 
dandelions. We imagine the spaghetti and 
garlic bread for dinner. Then we started to 
race to the door. The End. 

As Ellie read her story, she stood still and looked 
down at the paper she was holding. Her posture was 
open; her body upright and engaged. She read 
fluently and did not stumble over any words. 

Toward the conclusion of her story, she briefly 
looked up at the camera. When she completed her 
reading, she looked directly into the camera, 
bounced on her toes in a quick up-down motion, 
and smiled. 

There is a contrast between Elbe’s first draft and her 
final copy in her communicative modes as web as 
response to the story. The first draft is the result of a 
five-minute “braindrain” wherein students were 
asked to write about people or sounds in their 
community. The final version of Elbe’s community 
story was written after talking with her family, 
taking twelve pictures that were observed as 
negatives, and selecting the picture that represented 
her community to her. She used that picture to write 
the story and was positioned as a skillful writer. Elbe 
read with a quiet voice but without any of the 
movement and distractions from her earlier reading. 
Elbe did seem to look up for approval once toward 
the end, but she quickly returned to reading. When 
she was finished reading, Elbe looked up at the 
camera and gave a huge smile; in this final clip, she 
exudes confidence. 


In the first draft, Elbe was focused on the author’s 
style. Her writing featured repetition of words and a 
rhythmic style, which echoed Christopher Myers’s 
(1999) description in the story. Her integration of 
“dreaming” evokes images of strong memories of 
what she considers community. She incorporated 
creative language with her word choice and the 
sounds she included. The final draft reflects a 
conversation with her community that was 
influenced by her experiences outside of school. She 
integrated very specific images that we could see in 
photos and her attention to these images supported 
her storytelling. She also used the collective “we” 
along with specific conversations and references to 
her family, referring to memorable moments she 
spent with them. 

This literacy through photography assignment built 
on Elbe’s experiences outside of the classroom, and 
in both versions of the story, we see evidence of her 
strengths as a writer. In her initial draft, she 
capitalizes on the business, noisiness, and 
brightness of her community; our impression is that 
she is a part of a vibrant one. In addition, we can see 
that her father offered her guidance in this instance 
and encouraged her to think about how they defined 
community by what is important to the family (e.g., 
landmarks, people). In this way, Elbe was building 
on the funds of knowledge and the experiences 
available in her home (Gonzalez, Mob, & Amanti, 
2003). Our microanalysis revealed the nuances of 
the home-school relationship and its implications 
for Elbe’s literacy practices. 

Discussion and Implications 

We observed how important it was for Elbe to have 
the opportunity to guide her own meaning making. 
Elbe’s final product incorporated various aspects of 
her poetic voice, community knowledge, and 
understanding of the assignment. We have three 
main implications from this research that include 
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the potential for multimodal learning in the 
classroom and the importance of video data analysis. 

Potential for Multimodal Learning in the 
Classroom 

Despite being engaged and interested in school, Ellie 
struggled to meet local and state literacy 
benchmarks. She enjoyed peripheral success; 
however, the Literacy Through Photography project 
allowed Ellie to bring aspects of her community into 
the language arts classroom and showcase her 
writing skills. The visual image provided more than 
a fixed representation of her community; her 
photograph opened semantic pathways (Halliday, 
1978) and allowed her to build meaning across 
modes. For Ellie, we see that creating school texts or 
responding to teacher-directed 
inquiry was a challenge. In the 
first clip, we observed that her 
spoken language was slow when 
trying to meet her teacher’s 
requirement for sound in her 
work (see Figure 1). We viewed 
this again in the second clip 
when Angela asked her to give 
three words that describe her 
community (see Figure 3). 

However, when Ellie was able to 
rely on photographs, her speech 
was more fluent, and her writing was more complex. 

We see potential for students to develop their 
writing abilities through the use of photography or 
other multimodal means in open-ended ways where 
students have time and space to reflect and develop 
their own meanings. Children construct the world 
with semiotic resources that are available (Gilje, 

2010; Kyser, 2013). The opportunities that students 
have to respond and participate within literacy 
events of the classroom play an important role in the 
opportunities they have in their lives. In this 
particular case, the photographs supported Elbe’s 


meaning-making. Photography provided an 
opportunity for Ellie to participate in new literacy 
practices as she integrated values, meanings, and 
perspectives from her community. 

Video data analysis 

Video data analysis offers insights into struggling 
readers’ behaviors that may otherwise be missed. 

For example, our microanalysis revealed times when 
Ellie demonstrated modal density; therefore, at least 
three communicative modes were active (i.e., 
spoken language, gesture, gaze, posture, head 
movement). At such times, Ellie was the most 
talkative and constructed more complex meaning. 
We noted that during times of modal dissonance, 
Ellie struggled to make meaning. 

Microanalysis has the potential 
to help researchers pinpoint 
times when modal dissonance 
occurs and lead to a better 
understanding of how students 
do and do not construct 
meaning. For example, 
researchers may ask: 

• Are struggling students able 
to convey complex meaning 
without modal density? 

• Do struggling students rely 
on artifacts when formulating answers to 
open-ended questions? 

Microanalysis may offer literacy researchers a new 
tool for investigating ways to meet the needs of 
struggling readers and writers. While this type of 
analysis may be impractical for practicing teachers, 
it demonstrates the need for further understanding 
of body language, gaze, and attention of students. 

Limitations 


r "N 

“The opportunities that 
students have to respond 
and participate within 
literacy events of the 
classroom play an 
important role in the 
opportunities they have in 
their lives.” 

V. J 
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Although we followed a rigorous process when 
conducting this microanalysis, we recognize that 
this study has limitations. In particular, our 
perceptions as white middle-class women may have 
led us to make assumptions about Ellie that differed 
from how she would have positioned herself. 
Moreover, this single-subject case limits our ability 
to generalize the findings. These lessons learned 
about Elbe’s progress within a multimodal project 
express her communicative modes, which are not 
necessarily transferrable to other readers. We realize 
that since one researcher on this team collected 
data, this could represent a potential limitation of 
the study. However, our team spent extensive time 
discussing various modes, analyzing videos of 
multiple students from the classroom, and 
corroborating each other’s analyses. The 
collaboration ensured an in-depth focus that could 
not have been achieved by a single researcher. 
However, we maintain that increasing knowledge of 
incidents of modal density and dissonance (Norris, 
2004) exhibited by other readers who, like Ellie, are 
marginalized and separated from academic success 
may yield rich understandings of how to support all 
students. 


Conclusion 

The findings of this study expand upon extant 
research and demonstrate how a “struggling” 
student can use multimodal texts to transition to 
“success.” We hope educators can see the strengths 
of such transformations and provide opportunities 
for multimodal engagement in their classrooms or 
learning contexts (Siegel, 2006). The findings of this 
study may encourage further understanding of how 
multimodal approaches benefit all learners, 
including those who struggle. Research on 
multimodal methods of teaching and learning 
demonstrates how many ways of understanding 
response and engagement are being re-visioned as 
we recognize the impact of expanding options for 
making meaning. Findings from this study support 
expansive practices that elevate students’ reading 
and writing skills through meaningful multimodal 
experiences. 
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